Search for: All records

Extreme heat puts tremendous stress on human health and limits people’s ability to work, travel, and socialize outdoors. To mitigate heat in public spaces, thermal conditions must be assessed in the context of human exposure and space use. Mean Radiant Temperature (MRT) is an integrated radiation metric that quantifies the total heat load on the human body and is a driving parameter in many thermal comfort indices. Current sensor systems to measure MRT are expensive and bulky (6-directional setup) or slow and inaccurate (globe thermometers) and do not sense space use. This engineering systems paper introduces the hardware and software setup of a novel, low-cost thermal and visual sensing device (MaRTiny). The system collects meteorological data, concurrently counts the number of people in the shade and sun, and streams the results to an Amazon Web Services (AWS) server. MaRTiny integrates various micro-controllers to collect weather data relevant to human thermal exposure: air temperature, humidity, wind speed, globe temperature, and UV radiation. To detect people in the shade and Sun, we implemented state of the art object detection and shade detection models on an NVIDIA Jetson Nano. The system was tested in the field, showing that meteorological observations compared reasonably well to MaRTy observations (high-end human-biometeorological station) when both sensor systems were fully sun-exposed. To overcome potential sensing errors due to different exposure levels, we estimated MRT from MaRTiny weather observations using machine learning (SVM), which improved RMSE. This paper focuses on the development of the MaRTiny system and lays the foundation for fundamental research in urban climate science to investigate how people use public spaces under extreme heat to inform active shade management and urban design in cities. 

Lensless imaging is a new, emerging modality where image sensors utilize optical elements in front of the sensor to perform multiplexed imaging. There have been several recent papers to reconstruct images from lensless imagers, including methods that utilize deep learning for state-of-the-art performance. However, many of these methods require explicit knowledge of the optical element, such as the point spread function, or learn the reconstruction mapping for a single fixed PSF. In this paper, we explore a neural network architecture that performs joint image reconstruction and PSF estimation to robustly recover images captured with multiple PSFs from different cameras. Using adversarial learning, this approach achieves improved reconstruction results that do not require explicit knowledge of the PSF at test-time and shows an added improvement in the reconstruction model’s ability to generalize to variations in the camera’s PSF. This allows lensless cameras to be utilized in a wider range of applications that require multiple cameras without the need to explicitly train a separate model for each new camera. 

This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.